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Abstract 

The double beta decay of ^°^Mo to the 0^ and 2f excited states of ^^'^Ru is studied 
using the NEMO 3 data. After the analysis of 8024 h of data the half-hfe for the 

two-neutrino double beta decay of ^'^'^Mo to the excited 0^ state is measured to be 

Ty'^J = [5.7lQg(stat)ib0.8(syst)]-10^'^ y. The signal-to-background ratio is equal to 
3. Information about energy and angular distributions of emitted electrons is also 
obtained. No evidence for neutrinoless double beta decay to the excited 0^*" state 
has been found. The corresponding half-life limit is Tj^^2^(0+ — O^*") > 8.9 • 10^^ y 
(at 90% C.L.). 

The search for the double beta decay to the 2f excited state has allowed the 
determination of limits on the half-life for the two neutrino mode Ty'^J[0'^ — > 2f) > 

1.1- 10^1 y (at 90% C.L.) and for the neutrinoless mode T^^iO"^ 2+) > 1.6- 10^3 y 
(at 90% C.L.). 

Key words: double beta decay, neutrino, ^^'^Mo, excited state 
PACS: 23.40.-s; 14.60.Pq 



1 Introduction 

The main interest in double beta decay is connected with the neutrinoless 
mode (0z//9/9) as a probe for physics beyond the Standard Model of electroweak 
interactions. Its existence is connected with fundamental aspects of particle 
physics, for example, non-conservation of the lepton number, the existence 
and nature of neutrino mass, the existence of right-handed currents in the 
electroweak interaction, the existence of a massless Goldstone boson (the Ma- 
joron) and the existence of supersymmetry [1,2,3]). 

Interest in neutrinoless double beta decay has seen a significant rebirth in re- 
cent years after the evidence for neutrino oscillations was obtained from the 
results of atmospheric [4] and solar [5,6,7,8,9] neutrino experiments (see, for 
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example, the discussions in [10,11,12]). This observation of oscillations was re- 
cently confirmed by the KamLAND experiment with reactor antineutrinos [13] 
and by the new SNO result [14]. These results are impressive proof that neutri- 
nos have a non-zero mass. However, the experiments studying neutrino oscilla- 
tions are not sensitive to the nature of the neutrino mass (Dirac or Majorana?) 
and provide no information on the absolute scale of the neutrino masses, since 
such experiments are sensitive only to neutrino mass-squared difference Am^. 
The detection and study of Oi//3/3 decay is the only practical method to clarify 
the following characteristics of the neutrino (see discussions in [15,16,17,18]): 
(i) neutrino nature (is the neutrino a Dirac or a Majorana particle?), (ii) ab- 
solute neutrino mass scale (a measurement or a limit on mi), (iii) the type of 
neutrino mass hierarchy (normal, inverted, or quasi-degenerate), (iv) CP vio- 
lation in the lepton sector (measurement of the Majorana CP-phases, which 
requires high precission neutrino oscillation data and reliable nuclear matrix 
elements) . 

In connection with the Oz//5/3 decay, the detection of double beta decay with 
emission of two neutrinos (2z//5/3), which is an allowed process of second order 
in the Standard Model, provides the possibility for experimental determina- 
tion of the nuclear matrix elements (NME) involved in the double beta decay 
processes. This leads to the development of theoretical schemes for nuclear 
matrix-element calculations both in connection with the 2z//?/? decays as well 
as the 0uf3f3 decays (see, for example, [19]). At present, 2uf3f3 to the ground 
state of the final daughter nucleus has been measured for ten nuclei: ^^Ca, 
^^Ge, 82Se, 96Zr, ^o^Mo, i^^cd, ^^^Te, ^^^Te, ^^^Nd and ""^^U (a review of re- 
sults is given, for example, in Refs. [20,21]). 

The /?/? decay can proceed through transitions to the ground state as well as to 
various excited states of the daughter nucleus. Studies of the latter transitions 
allow supplementary information about (3/3 decay to be obtained. Because 
of smaller transition energies, the probabilities for /5/9-decay transitions to 
excited states are substantially suppressed in comparison with transitions to 
the ground state. The first exclusive studies of I3j3 — ^ coincidence experiments 
have been done on ^°°Mo about 15 year ago [22]. But as it was shown in 
Ref . [23] , by using low-background facilities utilising High Purity Germanium 
(HPGe) detectors, the 2vf3l3 decay to the O^" level in the daughter nucleus may 
be detected for such nuclei as -"^"^Mo, *^^Zr and ^^°Nd. In this case the energies 
involved in the j3j3 transitions are large enough (1903, 2202 and 2627 keV, 
respectively), and the expected half-lives are of the order of 10^° — 10^^ y. The 
required sensitivity was only reached for ^^'^Mo (the transition was detected in 
three independent experiments [24,25,26]) and ^^°Nd [28]. Recently additional 
isotopes, ^^Se, ^'^°Te, ^^^Cd and ^^Ge, have also become of interest to studies 
of the 2i/j3j3 decay to the level (see review in Refs. [21,29]). 

Theoretical estimates of the 2i//3/3 decay to a 2+ excited state have shown that 
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for a few nuclei (^^Se, s^Zr, 100^0, and ^^^Te) the half-lives can be ~ lO^^ - 
10^^ y [30,31]. This would mean that the detection of such decays becomes 
possible using the present and new installations in the future. 

It is very important to note that in the framework of QRPA models the be- 
haviour of NME(2z/) parameter gpp is completely different for transitions to 
the ground and excited (2+ and O"*") states [30,32]. This is why the decay to 
excited states may probe different aspects of the calcTilation method than the 
decay to the ground states. The Ouf3f3 transition to excited states of daughter 
nuclei, which provides a clear-cut signature of such decays, is worthy of special 
note here: in addition to two electrons with fixed total energy, there appears 
one (0+ — 2+ transition) or two (0+ — 0+ transition) photons, their energies 
being strictly fixed as well. In a hypothetical experiment detecting all decay 
products with a high efficiency and a high energy resolution, the background 
could be reduced nearly to zero. Possibly this idea will be used in future ex- 
periments featuring a large mass of the substance under study (as mentioned 
in Refs. [21,29,33]). In Ref. [34] it was mentioned that detection of this transi- 
tion will give us the additional possibility to distinguish the Oi//3/3 mechanisms 
(the light and heavy Majorana neutrino exchange mechanisms, the trilinear 
R-parity breaking mechanisms etc.). So the search for (3(3 transitions to the 
excited states has its own special interest. 

The ^°°Mo decay scheme is shown in Fig. 1. The decay to the O^" excited 

state of ^''^Ru is accompanied by the simultaneous emission of two electrons 
and two 7-rays of the energies of 540 keV and 590 keV. The emission of two 
electrons and of one 7-ray of energy 540 keV is the signature of the double 
beta decay to the 2i excited state. 
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Fig. 2. A schematic view of the NEMO 3 detector. 
2 NEMO 3 experiment 

NEMO 3 is a currently running experiment searching for neutrinoless double 
beta decay. The philosophy behind the NEMO Sdetector is the direct detection 
of the two electrons from (3j3 decay by a tracking device and a calorimeter [35]. 
A schematic view of the NEMO 3 detector is shown in Fig. 2. The NEMO 3 
detector has three main components, a foil consisting of different sources of 
double beta decay isotopes and copper, a tracker made of Geiger wire cells and 
a calorimeter made of scintillator blocks with PMT readout, surrounded by 
a solenoidal coil. The detector has the ability to discriminate between events 
of different types by positive identification of charged tracks and photons. 
The estimated sensitivity of this detector for Oi//3/3 decay, a function of both 
the detector design and the isotope type, is of the order of 10^^ y, which 
corresponds to an effective neutrino mass {niy) at the level of 0.1 - 0.3 eV. 



2.1 General description 

The NEMO 3 detector is cylindrical in design and is composed of twenty equal 
sectors. The external dimensions of the detector with shields are about 6 m 
in diameter and 4 m in height. NEMO 3 is based on the techniques tested on 
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previous incarnations of the experiment NEMO 1 [36] , and NEMO 2 [37] : 

The wire chamber is made of 6180 open octagonal drift cells which operate in 
Geiger mode (Geiger cells). A gas mixture of ~ 95% hehum, 4% ethyl- alcohol, 
1% argon and 0.15% water at 10 mbar above atmospheric pressure is used as 
the filling gas of the wire chamber. Each drift cell provides a three-dimensional 
measurement of the charged particle tracks by recording the drift time and 
the two plasma propagation times. The transverse position is determined from 
the drift time, while the longitudinal position is deduced from the difference 
between the plasma propagation times at both ends of the cathode wires. The 
average vertex position resolution for the two-electron events is at = 0.5 cm 
in the transverse plane of the detector and a; = 0.8 cm in the longitudinal 
plane. The Geiger counters information is treated by the track reconstruction 
program based on the cellular automaton algorithm, described in [38] . 

The calorimeter, which surrounds the wire chamber, is composed of 1940 plas- 
tic scintillator blocks coupled by light-guides to very low-radioactivity photo- 
multiplier tubes (PMTs) developed by Hammamatsu. The energy resolution 
FWHM/E of the calorimeter ranges from 14.1 to 17.6% for 1 MeV electrons, 
while the time resolution is of 250 ps at 1 MeV. 

The apparatus accommodates almost 10 kg of different double beta decay 
isotopes, comprising io°Mo (6914 g), ^^Se (932 g), ^^^Cd (405 g). ^'^^Tc (454 g), 
^5°Nd (34 g), ^^Zr (9 g), and ^^Ca (7 g). Most of these isotopes are highly 
enriched, between 95 - 99%, and are produced in the form of thin metallic or 
composite foils with a surface density of 30 - 60 mg/cm^. Three sectors are 
also used for external background measurement and are equipped respectively 
with pure Cu (one sector, 621 g) and natural Tc (1.7 sectors, 614 g of ""*Tc02). 
Some of the sources, including ^^'^Mo, have been purified in order to reduce 
their content of ^o^Tl (from 232Th and 228Th, and from 228Ra with half-fife of 
5.75 y) and ^^^Bi (from ^^^Ra with half-life of 1600 y) either by a chemical 
procedure [39], or by a physical procedure [35]. The foils are placed inside the 
wire chamber in the central vertical plane of each sector. The majority of the 
detector, 12 sectors, is mounted with 6.9 kg of ^°°Mo. 

The detector is surrounded by a solenoidal coil which generates a vertical 
magnetic field of 25 Gauss inside the wire chamber. This magnetic field allows 
electron-positron separation by measuring the curvature of tracks. The ambi- 
guity of the e+/e^ recognition based on the curvature reconstruction is 3% at 
1 MeV. 

The whole detector is covered by two types of shielding against external 7-rays 
and neutrons. The inner shield is made of 20 cm thick low radioactivity iron 
and stops 7-rays and slow neutrons. The outer shield, comprised of tanks filled 
with borated water on the vertical walls and wood on the top and bottom. 
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was designed in order to thermalise and capture neutrons. 

The experiment is located in the Frejus Underground Laboratory, the LSM ^ , 
at the depth of 4800 m water equivalent where the remaining cosmic ray flux 
is only 4.2 muons per m^ per day [40]. In May 2002 the detector took its 
first data which were used for the study of the detector's performance. Since 
February 2003, after the final tuning of the experimental set-up, NEMO 3 
has routinely been taking data devoted to double beta decay studies. The 
cahbration with radioactive sources is carried out every 6 weeks. The stability 
of the calorimeter is checked daily with a laser based calibration system [35] . 

The advantage of the NEMO 3 detector rests in its capability to identify the 
two electrons from decay and the de-excitation photons from the excited 
state of the daughter nucleus. The NEMO 3 calorimeter also measures the de- 
tection time of the particles. The use of appropriate time-of-fiight (TOF) cuts, 
in addition to energy cuts, allows an efficient reduction of all backgrounds. 



2.2 Background 

According to its origin, the background events in NEMO 3 can be divided 
into two categories. The first category is called internal background and has 
its origin inside the source foils. This background is mainly due to the presence 
of radioactive impurities in the foils. Particularly dangerous are /^"-emitters 
with high values hke ^^^Tl {Qp = 4.99 McV) and ^i^Bi (Q^ = 3.27 McV). 
These isotopes can produce j3j3-\ike events through three mechanisms: (i) a /?- 
decay accompanied by an electron conversion process, (ii) the Moller scattering 
of a /3-decay electron, and (iii) a /3-decay emission to an excited state followed 
by the Compton scattering of the de-excitation photon. 

Great care was taken in the production and subsequent purification of the 
enriched materials, as well as during the source foil fabrication itself and its 
installation into the detector to keep any background contamination to the 
minimum. The source foils were required to satisfy strict radioactivity lim- 
its [35]. 

In addition to the radioactive impurities, the 2i//3/3 (g.s. — > g.s.) decay is also 
a source of internal background for the 2i//3/3 decay to excited states signal 
because of bremsstrahlung radiation from the electrons. The tail of the Eee 
distribution from 2i//3/3 overlaps the window of interest, to some extent, for 
the 0z//9/9 decay search. Its contribution depends on the 2i'P/3 decay half-life 
and the energy resolution of the detector. 



Laboratoire Souterrain de Modane; http://www-lsm.in2p3.fr 
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The second category of background in the NEMO 3 experiment is called exter- 
nal background and is caused by electrons or photons generated everywhere 
else except within the foils. One component of this background is due to 
radioactive isotopes present in the detector construction materials. This con- 
tribution was minimised by a strict selection of low radioactivity materials 
during the NEMO 3 construction [35,41]. Another component comes from the 
laboratory walls and from radon present in the laboratory air. Background 
from cosmic rays is negligible thanks to the placement of the detector in the 
underground site. Neutron background is also negligible due to the two types 
of shields against neutrons and 7-rays (from neutron capture) and because of 
use of the magnetic field, which during data processing allows the elimination 
of e+e" pairs coming from the interaction of high energy photons with the 
source foils. 

The most bothersome external background contribution comes from radon. 
Radon is a highly diffusive radioactive gas and is outgased into the air from 
the rock walls of the LSM laboratory. It can enter the detector either through 
tiny gaps between the sectors, or through gas piping joints. The average radon 
activity in air of the LSM laboratory is around 10 - 20 Bq/m^, while the 
measured value of average volume activity of radon inside the NEMO 3 input 
gas from February 2003 to September 2004 was about 25 - 30 mBq/m^. Radon 
disintegrates to ^^^Pb via two a-decays. The latter, after a subsequent I3~- 
decay, generates ^^"^Bi. The decay of ^^"^Bi to ^^"^Po is generally accompanied 
by one electron and several photons, it has a recognisable signature consistent 
with a real event and can only be partially rejected by the TOP cuts. ^^^Po 
is an a-emitter (T1/2 = 163 //s) and decays to ^^°Pb. NEMO 3 records events 
with a delay time up to 700 /is in order to tag the a-particles from the ^^^Po. 



3 Analysis and results 

In this work we use the data taken from March 17, 2003 up to September 
23, 2004. This time corresponds to the stable period of the operation of the 
NEMO 3 detector before the installation of an anti-radon factory in order 
to reduce radon background. The efficient data collection time is 8024 hours 
(0.92 y) and the total number of triggers was 2.2-10^. 

3.1 Particle identification in NEMO 3 

The NEMO 3 detector is capable of identifying electrons, positrons, photons, 
and a-particles by unambiguous topological signatures. We define them in the 
following way: 
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TOP VIEW SIDE VIEW 




Fig. 3. Top (left) and side (right) views of a reconstructed event candidate selected 
from data. Two electrons of 424 and 286 keV energy hit fired isolated scintillators. 
They have a common origin in the source foil. The remaining fired scintillators 
compose three 7-clusters of 405 (175+230) keV, 341 (152+189) keV and 81 keV. 
The two clusters of highest energy are due to 7-rays emitted in coincidence with 
the electrons. The third cluster is most probably due to the rescattering of one of 
these 7-rays. In this event the total deposited energy is 1537 keV. 

• An electron coming from the source foil is recognised as a track going an- 
ticlockwise ("negative" track curvature), found by the track reconstruction 
program and pointing to a fired scintillator. The scintillator has to be iso- 
lated from all other fired scintillators. 

• A positron is defined like an electron but with the opposite ("positive") 
track curvature. 

• A photon is defined as a group of one or more fired scintillators, in the same 
region of the detector, unassociated with any track. It is required that all 
the scintillators in the group have to be fired close together in time in order 
to exclude a random coincidence of hits of different origin. 

• An a-particle is defined as a group of delayed Geiger cell hits, which are in 
time within 1.5 /is and close to each other in space. It is required that an 
a-particle has to be close to the event vertex or to the electron (positron) 
track. 

An example of a typical real event, selected for the study of the double beta 
decay of ^^''Mo to the excited 0^ state, is shown in Fig. 3. 

3.2 Background estimation methods 

Two different methods of background estimation are used here to evaluate 
the decay half-life: the first one is based on Monte-Carlo (MC) simulations 
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(Method 1) [42], while the second one is based on event analysis in selected 
non- Molybdenum foils (Method 2). 

The Method 1 requires the knowledge of activities of all the background 
sources. These activities have been measured by the HPGe detectors in the 
LSM laboratory and by radon detectors [35]. A special data analysis allowed 
the verification of these radioactive contamination levels and further, to de- 
termine them with higher precision. 

The presence of non-Molybdenum foils in the NEMO 3 set-up allows them to 
be used for independent background estimation and to provide a cross check 
(Method 2) to the result obtained with the Method 1. The foils made of Cop- 
per, natural and enriched Tellurium, Selenium and Cadmium are used to this 
end. It is supposed that the contribution from double beta decay to the ex- 
cited states of other sources such as Tellurium, Selenium and Cadmium can be 
neglected due to the greater predicted half-lives than for Molybdenum [30,43]. 
Consequently, the events selected in the non-Molybdenum foils are expected 
to be composed only of the sum of the contributions of the internal foil con- 
taminations specific for each source and of the common external background, 
where the main background component is due to radon in the tracking wire 
chamber. Knowing the internal contamination values for each source foil [35], 
one can evaluate the contribution corresponding to the radon background. 
The number of events due to the internal foil contamination evaluated by MC 
calculation is found to be negligibly small compared to the number due to 
the radon background. The number of events from the radon background in 
non-Molybdenum foils is used to evaluate the radon background in Molybde- 
num by scaling in proportion to their foil surfaces. The correction due to the 
variation of the event registration efficiency for different detector parts has 
been also apphed. The recalculation coefficient is determined to be equal to 
1.7. 

For Method 2 it is not necessary to know the absolute level of the main 
background from MC simulation. This is an advantage of the method. However 
its accuracy is limited by low statistics of observed events in non- Molybdenum 
foils. 

3.3 Event selection 

In this section we described just the common selection criteria for the four 
channels of analysis. Other criteria specific for the different channels are given 
in the section devoted to the each analysis (see Sees. 3.4, 3.5, 3.6 and 3.7). 

Only events containing two electrons coming from the source foils and each 
depositing energy greater than 200 keV in the scintillators are selected. It is 



10 



required that each track be associated with a unique scintihator which nmst 
have a gain greater than 0.17 (kcV/ADC channel)^^. Only the tracks starting 
from the first two Geiger cell layers are kept. The track intersection with the 
source foil is called a track vertex and then the event vertex, i.e. the presumed 
double beta decay origin in the source foil, is the average position of the track 
vertices. Each event has to have its vertex on the source foil. The distance 
between the track vertex and the event vertex is set to be within 2 cm in the 
drift plane and within 4 cm in the vertical direction. In order to prevent the 
wrong determination of the charge of shorter tracks, the events with a track 
associated with the scintillators on the top or bottom endcaps of the detector 
close to the source foils are rejected. 

To determine whether the origin of the electrons in the event is the source foil 
( "internal" ) or outside the foil ( "external" ) , the time of flight of both particles 
is used to calculate the corresponding probability of each hypothesis Pint and 
-Pcxt [35]. The event is accepted as a double beta decay candidate if Pint > 0.01 
and Pext < 0.001. 

According to the decay scheme the number of 7-rays simultaneously emitted 
from the event vertex ("internal 7-ray") must be exactly two for the decay to 
the 0^ state and exactly one for the decay to the 2i state. The estimation 
of the probability that the 7-ray was emitted together with each electron 
from the source foil {internal hypothesis) or crossing the tracking chamber 
{external hypothesis) was based on the time-of- flight information. Only events 
with probabilities for electron-7 pairs Pint > 0.01 and Pext < 0.01 are selected. 

In order to suppress the background contribution due to the radon in the 
tracking chamber, events with ct-particles must be excluded. The tagging of a- 
particles allows the reduction of the radon background contribution by Ri40%. 

3.4 2u(3(3 decay of ^°°Mo to the Ol excited state of ^^^Ru 

Since in the double beta decay to the Oj^ excited state there are two electrons 
and two 7-rays, it is required that the number of 7-clusters must be greater 
or equal to two. This selection criterion is motivated by the fact that a 7-ray 
emitted from the source foil can be scattered off a scintillator depositing in it 
some energy and then hit another scintillator. This requirement allows us to 
increase the efficiency of the signal selection. The corresponding information 
about the proportion of events with one, two or more 7-clusters for the data 
and MC is given in Table 1. 

According to the decay scheme, the energy of each emitted 7-ray can not be 
greater than 590 keV. In order to obtain the biggest signal-to-background ratio 
(S/B), the upper hmit on 7-cluster energy is set to 550 keV and, at the same 
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Table 1 

The relative fraction of events with different numbers of 7-clusters for the data 
and MC simulation of the signal and the main background, radon, in the tracking 
chamber inside NEMO 3. 



Number of 
7-clusters per 
event 



Experiment 



MC simulation of 
2I//3/3 (0+ ^0^) 
of i°°Mo 



MC simulation, 
radon background 



1 
2 
3 

>3 



24.6% 
56.7% 
14.4% 
4.3% 



15.6% 
56.2% 
21.5% 
6.7% 



24.1% 
55.2% 
16.9% 
4.4% 
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Method 1, 10''"y 



Method 1,T1/2 in 10''"y 



Method 2, T1/2in10"y 



Fig. 4. Half-lives (T1/2) of the double beta decay of ^°°Mo to the 0~[ excited state 
of ^°°Ru, obtained by the two analysis methods for 1860 combinations of kinematic 
cuts. 

time, the energy of a cluster corresponding to the "internal 7-ray" cluster has 
to be greater than 100 keV. However, if the energy of an extra fired scintillator 
which is not an "internal photon" is more than 150 keV, the event is rejected. 
This requirement allows us to suppress events originating from an external 
7-ray that deposits a part of its energy in one scintillator and then interacts 
with the source foil imitating the signal process. 

160 events remained in the data after this event selection. The total number of 
background events defined by Method 1 and Method 2 is found to be 71.4 and 
68.4 events, respectively. According to both background estimation methods 
the background is mostly due to the radon in the tracking volume of NEMO 3. 
To suppress this radon component in the background, events with the cosine 
of the angle between the two photons less than 0.8 are selected (Fig. 5(e)). 
A total of 1860 combinations of cuts on the following kinematic parameters 
has been carried out: the upper limit for the energy sum of the two electrons 

the energy of each electron (E™*^""): from 600 keV to 1000 keV in steps of 
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Table 2 

The results for the 2i//3/3 decay to the 0^ excited state: the signal registration effi- 
ciency (e), signal-to-background ratio (S/B), total number of selected events (S-l-B) 
and statistical significance of the signal (Ncr = S/^/S + B) for the two analysis 
methods. Results for both methods are given for the same kinematic cuts on the 
variables: E°f^, E™^^, E"!;'S E°"^. 



method 


Kinematic cuts in 


keV 


£•10-4 


S/B 


S+B 


No- 




^ee 


j^max 


T-imin 












Method 1 


1200 


700 


500 


125 


8.1±1.0 


3.0 


50 


5.3 


Method 2 


1200 


700 


500 


125 


8.1±1.0 


3.2 


50 


5.4 



50 keV; the lower limit on the energy sum for two "internal 7-rays" (E™): 
from 300 keV to 800 keV in steps of 100 keV; the lower limit on the energy 
of each "internal 7-ray" (E™''^): from 125 keV to 250 keV in steps of 25 keV. 
For each combination of cuts the two methods of background estimation are 
used. For each, the results of the calculations of the half-lives for both analysis 
methods as well as the correlation between them are shown in Fig. 4. The two 
methods give almost the same average value of the half- life around 6-10^° y 
and are in a very good agreement (see Fig. 4). 

Finally, after the cut optimisation we have chosen the kinematic cuts providing 

the maximal value of the product S/B x e using Method 1, where e is the 
signal registration efficiency. The optimal set of cuts for this criterion is given 
in Table 2. 

The distributions of the decay characteristics of selected events for Method 1 
after applying optimal kinematic cuts are shown in Fig. 5(1) and the corre- 
sponding background component contributions evaluated by this method are 
shown in Table 3. In Fig. 5(11) the same spectra obtained using Method 2 
are present. The numbers of signal and background events after the cuts ob- 
tained for Method 1 are: 37.5±7.1(stat.)±5.2(syst.) and 12.5±2.9(syst.). For 
Method 2 the numbers are 38.2 ± 8.4(stat.) ± 4.7(syst.) and 11.8 ± 4.5(stat.) 
± 0.4(syst.). 

The estimation of the systematic uncertainty has been made taking into ac- 
count the following contributions: 

• The two electron detection inefficiency due to the inaccuracy of the GEANT 
simulation and inaccuracy of the tracking program is estimated to be less 
than 5%. It is determined by reproducing the activity of the calibrated 
source placed inside the NEMO 3 detector. 

• The efficiency calculation error due to the inaccuracy of the GEANT sim- 
ulation for 7-rays is found by comparing the MC simulation and data from 
the calibration source. For the two 7-channel it is estimated to be around 
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Fig. 5. The main characteristics of selected events for the study of 2u(5(5 decay of 
i00]y[Q ^\yQ Q+ excited state of ^"'^Ru after the final kinematic cut application 
for Method 1 (I) and for Method 2 (II): (a) the distribution of the cosine of the 
angle between the two electrons, (b) the energy sum of the two electrons, (c) the 
energy sum of the two 7-rays, (d) the total energy sum, (e) the distribution of 
the cosine of the angle between the two 7-rays, (f) the energy of each electron, 
(g) the energy of each 7-ray. The experimental points are shown in black, the MC 
simulation of the background in blue, the simulation of the signal process in green. 
The fixed positions of the kinematic cuts for the best result are presented with red 
lines (E^'^^ = 1.2 MeV, E^f^^ = 0.7 MeV, E™"^ = 0.5 MeV, E?^'"^ = 0.125 MeV). 
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Table 3 

The calculated number of events in the observed spectrum for ^'^'^Mo foils from each 
background component. Only the components leading to a number of expected 
events greater than 0.1 are given here. 



Background 
component 


(to g.s.) 


internal 

214Bi 


internal 

208 rpj 


^i^Bi (radon 
inside tracking 
chamber) 


^i^Bi (radon 
outside track- 
ing chamber) 


Number of 
events 


0.5 


0.2 


0.3 


11.2 


0.3 



10%. 

• The effect of the uncertainty in the energy calibration coefficients causes a 
2% contribution to the systematic error in the half-life. It has been estimated 
by the energy scale variation for the experimental data. 

• The effect of the single electron energy threshold for the two electron signal 

is estimated as 3%. 

• The effect of the uncertainty of the detector efficiency is evaluated from the 
comparison of results of the 2i'(5(3 decay to the ground state separately for 
metaUic and composite foils of ^°°Mo. It is estimated to be around 4%. 

• The uncertainty of the radon level measurement is 20% which gives a 6% 
error in the half-life. This uncertainty is inherent only in analysis Method 1. 
All other background uncertainties do not introduce any significant error 
into the evaluation of result. 

The total systematic error on the result of analysis Method 1 is 13.8%, and of 
the Method 2 is 12.4%. It is comparable with the spread of the half-lives for 
the 1860 kinematic cut combinations presented in Fig. 4. 

The final results of the half-life measurements for the two neutrino double 
beta decay of ^°°Mo to the O^" excited state of ^°°Ru for both methods are the 
following: 

7^/2^(0+ ^ Of) = 5.7tli{stat.) ± 0.8{syst.) ■ 10^° y (Method 1); 
7^/2^(0+ ^ Of) = 5.6tl\l(stat.) ± 0.7(syst.) ■ 10^° y (Method 2). 

3.5 Search for the 01/(3 P decay to the Oi excited state of^'^^Ru 

For the study of the neutrinoless double beta decay of ^°°Mo to the Of excited 
state of ^°°Ru, in addition to the selection criteria described in Sec. 3.3, the 
energy of each electron is required to be less than 1600 keV and the energy of 
each photon (E^) to be greater than 125 keV and less than 600 keV according 
to the particle energy distributions for the Ou mode. 
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Fig. 6. Scatter plot of the sum of the 7-rays energy versus the energy sum of the 
electrons a) for the data (in black) and b) for the MC background (in blue). The 
simulated OuPP to the Of excited state signal ((a), in green) is also shown with an 
arbitrary scale. The bins with ss/^b > {^s/^b) are shown with a black frame. 

The two-dimensional distribution of the energy sum of the two 7-rays (-^77) 
and the energy sum of the two electrons (-Eee) is shown in Fig. 6. The expected 
signal (es) and the background (eb) efficiencies can be found for each bin 
of this distribution from the MC simulation. The signal-to-background ratio 
(es/^b) may be calculated for each bin, and their average value {es/eb) among 
all bins is found. It is reasonable to consider only the bins with a significant 
Es- If £s/^B > {^s/^b) is required, the region searched is composed of 27 bins 
(see the Fig. 6). 6 experimental events are observed and 6.6 MC background 
events are expected. Comparing experimental data with the MC background 
there is no evidence for the Ou signal. 



To set the limit we apply a method based on the confidence level computation 
with small statistics (CLs method) [44] treating each bin of the distribution 
as independent search channel. The uncertainty in the number of background 
events has been set at 20% (mostly due to the radon level uncertainty), for 
the signal efficiency error we use 5%. Using the MCLIMIT program [45] the 
number of signal events excluded at 90% confidence level is calculated to be 
3.9. Taking into account the signal efficiency of 1.3% one may evaluate the 
limit on the Ou decay half-life: 

tI%\0^ 0+) > 8.9 ■ 1022y (at 90% C.L.). 



3.6 Search for the 2u(3(3 decay of ^^^Mo to the 2^ excited state of ^°^Ru 



This decay produces the emission of two electrons and one photon of 540 keV 
(Fig. 1). To study this decay channel we perform the same data selection as 
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for the decay to the OJ excited state but with the modified criterion that the 
number of fired scintiUators in event is required to be greater than or equal 
to 3. The energy sum of aU 7-clusters must be less than 550 keV. In order to 
select events in the energy region specific for this decay channel, we used the 
following kinematic cuts for event selection: the total detected energy must be 
greater than 1000 keV; the energy of the "internal 7-ray" must be greater than 
250 keV; the cosine angle between an electron and the 7-ray must be less than 
0.8. The latter cut is used in order to suppress the possible background from 
the double beta decay to the ground state accompanied by a 7-ray tangent to 
one of electrons emitted by the Bremsstrahlung process in the source foil. 

531 events result from this selection, 465 of them can be attributed to the 
background according to analysis Method 1. The main background contribu- 
tions according to the MC simulations come from the radon in the tracking 
chamber (38% of total background), 2z//5/3 decay to the 0^ excited state (15%), 
2vj3j3 decay to the ground state (15%), radon outside the tracking chamber 
(12%) and internal radioactive contamination of PMTs in ^'^^Th (12%) and 
^^^Ra (12%). The error on the signal (cr) is 43.2 and includes also the system- 
atic uncertainty of the background calculation. The statistical significance of 
the disagreement between MC and data is 1.5(7. Using the signal efficiency 
defined by the MC simulation (0.45%) the half-life limit of the 2v(5(5 decay of 
^°°Mo to the 2^ excited state of ^°°Ru is calculated to be: 

r,%"^(0+ ^ 2+) > 1.1 • 10'^ y (at 90% C.L.). 



3. 7 Search for the Oi^pp decay of ^°°Mo to the 2t excited state of ^^^Ru 



A similar analysis to that described in Sec. 3.5 has been performed to search 
for the neutrinoless double beta decay of "^^''Mo to the 2i excited state. After 
the preliminary analysis the events with the typical topology were selected 
and the two-dimensional distribution of the energy sum of the electrons E^e 
and the energy of the 7-rays E-y was analysed. 

Using the same approach as for the 2/90z/ (0"*" — >• Of) (see sect. 3.5) there are 31 
bins of the distribution with 63/ Sb ratio greater than the average as shown in 
Fig. 7. The signal efficiency is found to be 2.9% ^ , 6 events are observed and 
5.7 background events are expected. In the limit calculations the background 
uncertainty (20% mostly due to the radon level uncertainty) and the error on 
the signal efficiency (5%) have been taken into account. 4.9 signal events are 
excluded on 90% C.L. and the corresponding limit on the half-hfe is: 

^1/2^(0+ ^ 2+) > 1.6 • 10^3 y (at 90% C.L.). 

^ The N* mechanism of (3(3 decay [46] has been used for MC simulation. 
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Fig. 7. Scatter plot of the 7-ray energy versus the energy sum of electrons a) for the 
experimental data (in black) and b) for MC background (in blue). The simulated 
Quf3[5 to the 2^ excited state signal in arbitrary scale ((a), in green) is also shown. 
The relevant bins are shown with a black frame. 



4 Discussion 



The results of this analysis of the /3/3-decay of ^°°Mo to the excited states of 
^'^"Ru are presented in Table 4 along with previous experimental results and 
theoretical estimates for the half-lives. 

For the transition to the Oi" excited state of ^°°Ru the obtained value is in 
a good agreement with results of previous experiments. It has to be stressed 
here that in the present experiment with the NEMO 3 detector, all the decay 
products (two electrons and two 7-rays) were detected for the first time and 
hence all the useful information about the decay was obtained (total energy 
spectrum, single electron spectrum, single 7 spectrum, and all angular dis- 
tributions). In previous experiments either individual 7-rays [24,25], or the 
coincidence between the two emitted 7-rays [26] were detected but no infor- 
mation about electrons was obtained. 

One can compare the measured value with the theoretical predictions, which 
were obtained using QRPA-based models [49,31,55], and the pseudo-SU(3) 
model [51]. The present accuracy of the theoretical prediction is directly con- 
nected to the accuracy of the nuclear matrix elements (NME) calculations 
and usually is a factor of ~4 for the half- life (factor of ~2 for the NME). In 
view of this, it can be seen that there is a reasonable agreement between the 
experimental and theoretical values of the half-life. 

Recently an estimation of the half-life value for the 2v(3j3 (0^ — > 0]*") transition 
of ^°°Mo was carried out using the so-called single state dominance hypothesis 
(SSDH) [31,56] and the recent result for electron capture (EC) transition ^°°Tc 
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Table 4 

Experimental results and theoretical predictions for the half-life of the (3/3-decay of 



lO^Mo to the excited states of i^ORu. All limits (except one specifically mentioned) 
are at 90% C.L. Theoretical predictions for Ou mode are presented for (m,^) = 1 eV 
and (A) = 10-^ 



Transi- 

1 iou 


Ti/2, y 

(this work) 


Ti/2, y 

([previous works) 


Ti/2, y 

(ihcor. |")r('<iicl ions) 


OuPP; 

0+ ^2+ 


>1.6-1023 


>1.4- 10^2(68% C.L.) [47] 


6.8-103°((m^)) 
2.M027((A)) [48] 


2zy/3/3; 
0+-.2+ 


>l.l-102i 


>1.6-102i [24] 


3.4-1022 [49] 
(5.5-24)-1025 [51] 
1022 [50] 
2.1-1021 [31] 


Ou/313; 
0+ ^0+ 


>8.9-1022 


>6-1020 [52] 


(7.6-14.6)-1024 

((m,)) [53] 
2.6-1026((m^)) [54] 


2zy/3/3; 
0+^0+ 


[5.7;^J(stat)±0.8(syst)]-1020 


6.1+|i-1020 [24] 

[9.3l2-8(stat)±1.4(syst)]-1020 [25] 
[5.9tli (stat) ±0.6(syst)] • lO^o [26] 
>1.2-102i [27] 


(0.7 - l)-102i [55] 
2.1-1021 [49] 
1.5-1021 [51] 
1.5-1020 [31] 



^ looMo [57]: T[/2 {^^ Of) = 4.2-102° y [43] (with an accuracy of ± 50% 
because of such accuracy of the EC transition [57]). This value is in agreement 
with our result and if in the future the accuracy of the EC transition can be 
improved, then a comparison between experimental and estimated values will 
provide the possibility to check more precisely the SSDH prediction. 



Using the phase-space value G = 1.64-10 y ^ (for — 1.254) and the 
obtained half-life value, one obtains the nuclear matrix element value (NME) 
for the 2iy(3(3 (0+ Of) transition, M(O^) = 0.103±0.011. One can com- 
pare this value with the NME value for the 2z/-transition to the ground state 
of iooRu, which was obtained also with the NEMO 3 detector, M(g.s.) — 
0.126±0.005 (here Tff^^ = [7.11±0.02(stat)± 0.6(syst)]-10is [58] and G = 
8.9-10-1^ y-i [30]). One can see that M(g.s.) is ~20% greater than M{Ot). 
Nevertheless, these values are very close and, taking into account errors and 
possible uncertainties, their equality can not be excluded. From the viewpoint 
of the theory it is important to confirm this difference or to rule it out, in 
particular to ascertain the validity of the SSDH. Future measurements with 
the NEMO 3 detector where results for both transitions (to the 0+ ground 
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and the Of excited states) will be improved, will address this further. 

The limit obtained for the 2i>(3P{0^ — > 2^) decay is comparable with the limit 
from Ref. [24] but is 1.5 times lower. In this channel a small excess of events is 
observed, which probably can be explained by a statistical fluctuation. Future 

measurements are needed to clarify the status of these extra events. The most 
optimistic theoretical prediction for ""^^^Mo gives T^^2''(0''' ~^ '^t) ~ 2-10^^ — 
10^^ y [50,31] for this type of decay which is not very far from the obtained 
limit. However, the estimation in the framework of the SSDH, which looks 
more reasonable, gives a higher value, T1/2 = 1.7-10^^ y [43]. 

Concerning the bound on the Ouf3f3 decay of ^°°Mo to 2i and 0^^ excited 
states of ^^^Ru the limits obtained in the present work are better than results 
of previous experiments by a factor of ~10 and ~100, respectively. Using the 
results of theoretical calculations from Refs. [53] and [54] for the OuPP decay 
of ^''^Mo to the of state of ^"^^Ru, one obtains a limit on the effective value of 
the Majorana mass, {m^) < 9.3 — 12.8 eV and 60 eV, respectively. It means 
that to reach a sensitivity to {m^) at the level of ~ of 0.1 eV, the sensitivity 
to the half-hfe has to be at the level of ~10^^ y. 

The Oi//5/9(0^ 2f) decay had long been accepted to be possible only because 
of the contribution of right-handed currents and it is not sensitive to the 
neutrino mass contribution. However, in Ref. [48] it was demonstrated that 
the relative sensitivities of (O"*" — > 2^) decays to the neutrino mass (111^) and 
the right-handed currents (rj) are comparable to those of OuPP decay to the 
ground state of the daughter nuclei. At the same time, the (O"*" — > 2^) decay is 
relatively more sensitive to (A). Using the limit of T1/2 > 1.6- 10^'^ y, the value 
for the NME from Ref. [48] and the phase-space value from Ref. [59] , one can 
obtain the limits (m^) < 6.5-10^ eV and (A) < 1.2-10-^. 



5 Conclusion 



The results from the NEMO 3 detector for ~1 year of measurement and with 
a higher level of background from ^^^Rn than expected, have been obtained. 
Data taking will be continued for a further ~5 years with an improved level 
of radon background. It means that the sensitivity of the experiment will be 
improved by a factor of ~ 5 — 10. For example, in the case of 2i//3/3(0+ — > 0^) 
decay ~ 200 events will be detected under conditions of a signal-to-background 
ratio of ~ 10 and an accuracy for the half-life value reduced by ~10%. The 
sensitivity to the 2vPP{0^ 2^) decay will be on the level of ~ 10^^ y and 
it could also yield a chance to detect this type of decay (see Table 4) . For the 
Ov transitions to the 2f and Of excited states the sensitivity will be ~ 10^^ y. 
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